Filtering Large Propositional Rule Sets While Retaining Classifier Performance

Data mining is the problem of inducing models from data. Models have both a descriptive and a predictive aspect. Descriptive models can be inspected and used for knowledge discovery. Models consisting of decision rules – such as those produced by methods from Pawlak’s rough set theory – are in principle descriptive, but in practice the induced models are too large to be inspected. In this thesis, extracting descriptive models from already induced complex models is considered. According to the principle of Occam’s razor, the simplest of two models both consistent with the observed data should be chosen. A descriptivemodel can be found by simplifying a complex model while retaining predictive performance. The approach taken in this thesis is rule filtering; post-pruning of complete rules from a model. Two methods for finding high-performance subsets of a set of rules are investigated. The first is to use a genetic algorithm to search the space of subsets. The second method is to create an ordering of a rule set by sorting the rules according to a quality measure for individual rules. Subsets with a particular cardinality and expected good predictive performance can then be constructed by taking the first rules in the ordering. Algorithms for the two methods have been implemented and is available for general use in the ROSETTA system, a toolkit for data analysis within the framework of rough set theory. Predictive performance is estimated using ROC analysis, and ten different formulas from the literature that can be used to define rule quality are implemented. An extensive experiment on a real-world data set describing patients with suspected acute appendicitis is included. In this study, rule sets consisting of six to twelve rules with no significantly different estimated predictive performance compared to full models consisting of between 400 and 500 rules were found. Another experiment confirms these results. In the experiments, statistical hypothesis testing was used to assert difference between performance measures derived from ROC analysis.